ci: OpenAPI contract validation (#37 phase 1)#44
Merged
Conversation
Add a zero-dependency drift check for the website-mirrored public OpenAPI spec (docs/openapi/qveris-public-api.openapi.json): asserts the file exists and is valid JSON, info.version is present, the core agent paths exist, and the response schemas the toolkit deserializes are present. Mirrors the ecosystem-validate workflow pattern (validator + regression test + path-scoped workflow). This is phase 1 only — type generation for MCP/Python SDK (phases 2-3) is intentionally deferred to separate PRs per the issue, so existing MCP, CLI, and Python SDK public APIs are unchanged. Closes #37 phase 1.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a zero-dependency OpenAPI contract validation script and a corresponding regression test suite to detect drift in the mirrored public API specification. The validator ensures that essential paths and component schemas required by the toolkit are present. Review feedback highlights the need for more portable path resolution using fileURLToPath to support Windows environments and recommends adding a check to verify that the parsed JSON is a valid object before accessing its properties.
- Use fileURLToPath(import.meta.url) for REPO_ROOT in both scripts (portable on Windows and paths with spaces). - Guard that the parsed spec is a non-null object before accessing properties, so `null`/array JSON yields a clean validation error instead of a TypeError crash. Covered by a new regression test.
Contributor
Author
|
Addressed all three Gemini comments in
|
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
A suggestion applied via the GitHub UI (cd0fa09) re-inserted the JSON.parse catch block, leaving an orphaned `} catch (error) {` that crashed CI with `SyntaxError: Unexpected token 'catch'`. Remove the duplicate; validator + 7 regression tests pass.
Contributor
Author
|
Fixed the failing Root cause: commit Removed the duplicated block (fix-forward, no force-push so |
vxwork
approved these changes
May 18, 2026
This was referenced May 18, 2026
linfangw
added a commit
that referenced
this pull request
May 18, 2026
…ase 2) (#48) * feat(openapi): generate checked-in MCP/Python contract types (#37 phase 2) Implements phase 2 of #37 / closes #46. Generated artifacts are a contract reference; hand-written MCP/SDK public models are unchanged (no runtime or package behavior change — backward compatible). - MCP/TS: `openapi-typescript@7.4.4` (pinned devDep) -> checked-in packages/mcp/src/generated/openapi.d.ts; `npm run gen:openapi` regenerates. - Python SDK: `datamodel-code-generator==0.26.3` (pinned dev extra) -> checked-in packages/python-sdk/qveris/generated/openapi_models.py (pydantic v2, py3.8 target) + package __init__ documenting regeneration. - CI: .github/workflows/openapi-types.yml regenerates both with the pinned generators and fails on `git diff --exit-code`, path-scoped to docs/openapi/** + the generators (mirrors openapi-contract.yml from #44). Verified: artifacts are byte-reproducible from the checked-in spec (empty diff on regen), the .d.ts typechecks under tsc --strict, and the Python models pass py_compile. * feat(contract): CLI + Python OpenAPI contract guards (#37 phase 3) (#49) * feat(contract): adopt generated OpenAPI types — CLI + Python guards (#37 phase 3) Implements phase 3 of #37 / closes #47. First low-risk adoption step; public MCP/CLI/Python SDK APIs unchanged (additive tests + CI only). - CLI (JS/MJS): packages/cli/test/openapi-contract.test.mjs asserts every endpoint+method the CLI calls (kept in sync with src/client/api.mjs: POST /search, /tools/by-ids, /tools/execute; GET /auth/credits, /auth/usage/history/v2, /auth/credits/ledger) exists in the mirrored public OpenAPI spec — the issue's "CLI contract tests before type gen". - Python SDK: tests/test_openapi_contract.py starts *consuming* the generated qveris.generated.openapi_models as a drift guard (core contract models present + pydantic + smoke roundtrip). Hand-written qveris.types stays the public surface; field alignment stays gradual by design. - CI: .github/workflows/contract-tests.yml runs both on PRs (the existing cli/python *-publish workflows only fire on release tags, so without this the guards would not gate PRs). Verified locally: CLI `node --test` 7/7 pass; Python pytest 11/11 pass. Builds on #46/#48 (needs the generated artifacts) — merge after #48. * fix(contract): load generated models standalone + robust shape check (#47) CI python-contract failed: importing qveris.generated.openapi_models pulled qveris/__init__.py (httpx not installed in the minimal contract job). Load the generated file by path so the guard only needs pydantic. Replace the instantiation smoke with a model_fields shape assertion (the generated file's __future__ annotations + constrained types need model_rebuild() to validate; the field map is the stable contract signal regardless). * Update packages/mcp/package.json Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements Phase 1 of #37.
Summary
scripts/validate-openapi-contract.mjs— zero-dep check on the website-mirroreddocs/openapi/qveris-public-api.openapi.json: file exists + valid JSON,info.versionpresent, core paths present (/search,/tools/by-ids,/tools/execute,/auth/usage/history/v2,/auth/credits/ledger), and the component schemas the toolkit deserializes (search/execute/usage/ledger families).scripts/test-openapi-contract.mjs— regression tests (valid spec passes; missing file / bad JSON / missing version / missing path / missing schema all fail)..github/workflows/openapi-contract.yml— runs both, path-scoped todocs/openapi/**+ the scripts + the workflow, mirroringecosystem-validate.yml.Scope / non-goals
Phase 1 only. Type generation for MCP (
openapi-typescript) and Python SDK (datamodel-code-generator) — Phases 2–3 — are intentionally deferred to separate PRs per the issue's "small PRs by phase" guidance. No package/runtime changes; MCP, CLI, and Python SDK public APIs are untouched and backward compatible.Test plan
node scripts/validate-openapi-contract.mjs→ OK against the checked-in specnode scripts/test-openapi-contract.mjs→ 6/6 regression tests passFollow-ups (separate issues/PRs): Phase 2 (generated TS/Python artifacts +
git diff --exit-codeCI), Phase 3 (gradual internal adoption).